Solid state silicon image sensors became ubiquitous in the recent years. The first high quality image sensors were fabricated in a CCD technology which is nowadays being more and more replaced by CMOS technology. While typical CMOS based image sensor sensitivity is still lower than CCD based sensors, the integration advantages that CMOS technology can provide make it a technology of choice for today's image sensors. Besides the integration, CMOS technology provides building blocks for active pixels which can extend the sensing capabilities of the image sensors beyond just imaging such as e.g. distance measurement, night/fog vision using high speed shutter image sensors, etc.
For machine vision applications, despite increasing resolution, increasing dynamic range of image sensors and increasing computational power of CPUs, it is impossible for a machine to extract distance information from a single 2D-image.
One solution is offered by so called stereo-vision systems. In traditional stereo vision, two cameras, displaced horizontally from one another, are used to obtain two differing views of a same scene, in a manner similar to human binocular vision. By comparing these two images, the relative depth information can be calculated. However, this requires two cameras and a powerful processor.
Another solution is offered by so called “time-of-flight 3D cameras”, which provide depth information about the scene. Time-of-flight 3D cameras typically use a near-infrared (NIR) (invisible to humans) modulated or pulsed light source to illuminate the scene, and the reflected near-infrared light is detected by high bandwidth time-of-flight pixels. The time it takes for a light signal to travel from the light source (emitter) to the object and back is proportional to the distance between the emitter and the object. This time delay between the emitted signal and the detected signal, also called “time-of-flight (TOF)” or “round-trip time (RTT)”, is usually estimated by frequency domain techniques (e.g. demodulation) or time domain techniques (e.g. correlation). Such “time-of-flight pixels” and corresponding processing circuitry exist, and are known in the art.
Although it is possible to get an image from a (pure) time-of-flight camera, the quality of such an image would be much inferior to the image quality of a typical image sensor as used e.g. in digital cameras. One of the shortcomings is image resolution. Typical time-of-flight pixels are typically much larger than typical image sensor pixels to increase the sensitivity. Another problem is that image sensors and time-of-flight cameras set different requirements towards the optics. A time-of-flight camera typically requires only a narrow band of near-infrared light to reduce the shot noise and early saturation (e.g. due to sunlight), whereas an image sensor requires only visible light.
In order to get both image information (e.g. intensity and/or color information) and distance information, some prior art systems combine an image camera and a 3D-camera, by implementing them as two separate systems. However this requires a high system cost.
US2013/0234029 discloses an image sensor for two-dimensional and three-dimensional image capture, comprising visible light photodetectors embedded in a first substrate, and TOF photodetectors embedded in a second substrate, which two substrates are combined by a bonding technique to form a single chip. However, the cost of such a sensor is relatively high.
US2007051876A1 discloses an imager in which a visible image and an infrared image can be independently and simultaneously obtained. The solution proposed requires two substrates oriented perpendicular to each other, and a mirror oriented at 45° with respect to these substrates.